Papers by Pankaj Kumar Sharma
SAJA: A Simple Approach to Judge Alignment for LLM-as-a-Judge (2026.acl-industry)
Copied to clipboard
| Challenge: | Current approaches to evaluate text at scale require multiple calls and per-dataset prompt tuning. |
| Approach: | They propose a model-agnostic approach to evaluate judge alignment that uses a lightweight calibration head. |
| Outcome: | a new model with SAJA matches more complex systems across four evaluation paradigms . it outperforms uncalibrated models on MT-Bench pairwise preference and competitive performance on five classification benchmarks compared to uncalibred models . |